Cardiovascular survey methods THIRD EDITION

ion of medical data from available records will determine whether the usualrecord provides clear and accurate data. If it is a laboratory test, blood, urine or othersamples can be collected from volunteers to determine whether the handling ofsamples and measurements are possible. The instrument is now ready to test for reliability, repeatability and validity. Differ-ent interviewers may interview the same individual over a short period of time. Forrecord abstraction, different abstractors may collect the data from the same record totest that the procedure is being done in an identical manner. For laboratory tests,split, blinded samples are used to determine the reproducibility of the sample han-dling and of the chemical tests. In each instance, an adequate sample is drawn tocheck that measurements made on the same individual or sample over a short periodof time produce the same result. Of particular interest is the stability of measurements over time. A particular test isrepeated after some predefined interval.A blood or urine test, for example, shouldgive the same result before or after storage. Validation of historical information such as clinical records or death certificates isperformed by obtaining more detailed information from diverse sources. Forexample, validation of hospital discharge diagnoses may involve a review of surgi-cal reports and of pathological reports, and interviews with physicians or otherinformants. Laboratory tests are frequently employed to confirm the presence or absence of adisease or risk factor. While no one test can provide certainty about disease status, itis clearly desirable to choose diagnostic instruments that offer the highest probabil-ity of accurate classification with the lowest individual inconvenience and economiccosts. Whether the test is performed as part of a study or the test's results collectedfrom available clinical records, similar questions must be asked of the test's validityand reliability. The validity or accuracy of a new test can be best established by "independent" com-parison with a reference "gold standard, namely a test that clearly identifies thedisease or characteristic under examination. Such a "gold standard is usually morecostly or complex than the survey measurement under consideration and is thereforegenerally unsuitable for large surveys. Validation may directly evaluate the diseaseprocess (e.g. a biopsy) or physiology (e.g. intra-arterial blood pressure measurement).Where no "gold standard exists, a composite standard of several measurements, oran expert consensus based on the results of several types of tests, is used. However,care should be taken to ensure that the observers interpreting the test are blinded tothe results of the "gold standard". The test being evaluated should also not be a partof the composite standard, as this will produce a bias towards agreement betweenresults of the two tests. 6. CONDUCTING THE RESEARCH The results of the test should also not influence the extent to which the standard testis likely to be applied. "Verification bias" or "work-up bias" occurs when a test isrestricted to patients in whom the disease status is already known. In such cases, themagnitude of the bias is directly proportional to the association between the diseaseand the result of the test under study. For example, a study validating echocardio-graphy as a decision-making instrument for cardiac surgery in valvular heart diseasemight report excellent diagnostic agreement between the findings at echocardiogra-phy and at surgery. However, this would not consider the fact that surgery has beenperformed only in those selected by echocardiography and that surgery might havebeen denied to some deserving patients (false-negatives not identified). It is impera-tive that the standard be uniformly applied to all persons in the study. For example,Goldschlager et al. (8)studied the accuracy of treadmill tests in the diagnosis of coro-nary artery disease using coronary angiography as the "gold standard. They included80 "healthy" male volunteers, who were presumed normal on the basis of clinicalevaluation and treadmill testing, in their category of true negative patients eventhough no angiography had been performed. The assumption that all unverifiedpatients are disease-free leads to false-negatives being classified as true-negatives. When a "gold standard is available, it is best to categorize test results into true-positives (disease present by both tests), false-positives (disease present only by thetest but not by the "gold standard), true-negatives (disease absent by both) andfalse-negatives (disease absent by the test but present by the "gold standard) is bestdone by constructing a 2 X 2 table (Figure 6.2). It is also essential to evaluate the reproducibility of test interpretation by differentobservers. Wide disagreement between observers in interpreting the same set of testresults renders the test useless for clinical or epidemiological application. Two or moreobservers should independently evaluate the test results without having access to theclinical data. If the test results are dichotomously classified, a chance-corrected indexof agreement (Kappa) should be calculated. If the data are continuous (as in a scoringsystem), intra-class correlation coefficients are the preferred method of evaluation. Pilot studiesA pilot field study to test the entire data collection protocol and schedule is neces-sary before active data collection begins. This differs from a pilot study of new mea-surement instruments, and it is best done on a population similar to the intendedpopulation of the actual survey, but involving only a small number of subjects. Theobjective of the pilot field study is to check that the data collection system is inte-grated and flows smoothly, thus optimizing the use of time by both the staff and par-ticipants. Pilot studies serve to reduce problems when the actual survey begins, andallow evaluation of the feasibility and acceptability to both staff and subjects of theplanned survey. To perform the pilot the various data collection instruments and testsare first ordered sequentially.A commonly used order of tests is shown in Table 6.3. CARDIOVASCULAR SURVEY METHODS